Start
- The numeric files has few additional features, but there is no description of the features so I will just go ahead with the file with both numeric and categorical features
EDA
Note: I have only done the EDA to answer the asked questions and to explore simple bivariate predictor-target relationships. I have not done any EDA for the purpose of feature engineering or feature selection.
Missingness
So, no missing data. Yayyy!
Credit Worthiness
Before going into exploring relationship of predictors with the target, let’s first clearly define the target
Credit worthiness for a group of observations can be measured by Good/Total proportion. Higher the proportion, higher the credit worthiness
Credit History
Question: Would a person with critical credit history, be more credit worthy?
Again, let’s first define what critical means. In the absence of any concrete definition, I will assume ‘critical’ roughly means more existing credits i.e. it increase from A30 to A35
Critical has positive association with credit worthiness
Age
Q. Are young people more creditworthy?
checking_account_status duration_in_months credit_history
0 0 0
purpose credit_amount savings_account_status
0 0 0
present_employment_since installment_as_percent_of_income marital_sex_type
0 0 0
role_in_other_credits present_resident_since assset_type
0 0 0
age other_installment_plans housing_type
0 0 0
count_existing_credits employment_type count_dependents
0 0 0
has_telephone is_foreign_worker is_credit_worthy
0 0 0
The distributions are quite overlapping. But there are more young in “Bad” compared to “Good”, and that is also visible in the difference in means.
So, young people seem slightly less credit worthy.
But let’s break the age into groups to see finer details
“Bad” is quite low for the (34, 39] age group
Credit Accounts
Q. Would a person with more credit accounts, be more credit worthy?
I am assuming more credit accounts is same as “Number of existing credits at this bank” i.e. ‘count_existing_credits’
`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text
Data is too unreliable to say anything on the relationship between no. of credit accounts and credit worthiness
Other continuous features
I will define continuous as any faeture having more than 10 unique values. This definition suits the visualization purpose.
Target vs Continuous Features
So, “duration_in_months” seems important, “credit_amount” not
Other categorical features
Target vs Categorical Features
[1] “credit_history” “age” “count_existing_credits”
- Cat. features which seem definitely important: “checking_account_status”, “purpose”, “present_employment_since”, “asset_type”, “other_installment_plans”, “housing_type”
- Cat. features where there is high uncertainity: “savings_account_status”, “installment_as_percent_of_income”, “is_foreign_worker”
- Cat. features which don’t: “marital_sex_type”, “role_in_other_credits”, “present_resident_since”, “employment_type”, “count_dependents”, “has_telephone”
Feature Engineering & Selection
As mentioned earlier I didn’t do any EDA from featre engineering perspective. So, there is no feature engineering.
For feature selection too I have not relied on my EDA because bivaraite analysis is not enough to figure out important features. There are better automated techniques. I have used Boruta, which I have found to be the best feature selection technique almost always. Below is how the Boruta plot looks like:
`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text`summarise()` ungrouping output (override with `.groups` argument)
Ignoring unknown aesthetics: text
Selected features are:
#### checking_account_status
#### purpose
#### savings_account_status
#### present_employment_since
#### installment_as_percent_of_income
#### marital_sex_type
#### role_in_other_credits
#### present_resident_since
#### assset_type
#### other_installment_plans
#### housing_type
#### count_existing_credits
#### employment_type
#### count_dependents
#### has_telephone
#### is_foreign_worker
















Features removed are:

This perfectly matches with our EDA too
Modeling
Strategy
It is worse to class a customer as ‘Good’ when they are ‘Bad’, than it is to class a customer as bad when they are good.
Let ‘Good’ be the positive class, and ‘Bad’ be the negative class. So the above statement will translate to:
False Positives (FPs) are more expensive than False Negatives (FNs)
Such cases fall under **Cost Sensitive Learning" strategy, and followong sub-strategies can be followed decided under it:
Strategy Options
- Modeling Strategies for cost sensitive learning
- Change cost function
- Change the function itself
- the main function
- penalty component
- Change function parameters
- oversample positive class
- synthetic sample generation (like SMOTE)
- give more weight
- undersample sample negative class
- Optimize thresholds that are used for converting output probabilities into class labels - valid only for models which output probabilities
- Ensembling
- Evaluation Strategies for Cost sensitive classification
- Favour Precision over Accuracy or Recall
- Give weights to different buckets in confusion matrix, and use that to construct a custom evaluation metric
Options that I will explore
Models
I will try the following three models: - Logistic Regression - Boosted Trees: GBM - Random Forest
Modeling Strategy
- Will optimize thresholds for all the models
- give more weight to positive class, I will tune the weighing parameter: will do this only for GBM, just to showcase
Evaluation Strategy
I will go with a Custom evaluation metric:
I have assigned follwing weights to different buckets of the confusion matrix to penalize each bucket differently
[1] "marital_sex_type" "present_resident_since" "count_existing_credits" "employment_type"
[5] "count_dependents" "has_telephone" "is_foreign_worker"
There is no particular reason for these values, just their relative differences are important because they penalize FPs more than FNs. PLus, I am rewarding TPs (True Positives)
Now, the custom metric is just the normalized sum-product of these weights and the confusion matrix of the model. Let’s call it “credit_cost”.
Splitting
I have 80:20 splitting. For validation, I will be using cross-validation wherever required.
Baseline
I am taking baseline as predicting everybody as "Good’
Train credit_cost
Reference
Prediction Good Bad
Good -0.4 1
Bad 0.2 0
Test credit_cost
Logistic Regression
I have just trained a simple Logistic Regressison without any regularization, class weighing, or hyperparameter tuning. This is supposed to be a ML model baseline.
Train Results:
Test Results:
|
| | 0%
|
|==================================================================================================| 100%
Test/Validation dataset is missing column 'employment_type': substituting in a column of NaN
|
| | 0%
|
|==================================================================================================| 100%
Test/Validation dataset is missing column 'employment_type': substituting in a column of NaN
Boosted Trees - GBM
Here, I have weighed the classes to favour “Good”. Weight has been found by tuning via grid search using 5-fold cross validation. I have not tuned other hyperparameters.
Train Results:
Test Results:
|
| | 0%
|
|==================================================================================================| 100%
|
| | 0%
|
|==================================================================================================| 100%
Random Forest
Here, I have tuned other hyperparameters via random search using 5-fold cross validation. I have not done class weighing here.
Train Results:
H2O Grid Details
================
Grid ID: drf_grid_11
Used hyper parameters:
- col_sample_rate_per_tree
- max_depth
- mtries
- ntrees
- sample_rate
Number of models: 7
Number of failed models: 0
Hyper-Parameter Search Summary: ordered by increasing logloss
Test Results:
|
| | 0%
|
|==================================================================================================| 100%
|
| | 0%
|
|==================================================================================================| 100%
Comparison
Confusion Matrix and Statistics
Reference
Prediction Good Bad
Good 535 52
Bad 26 189
Accuracy : 0.903
95% CI : (0.88, 0.922)
No Information Rate : 0.7
P-Value [Acc > NIR] : < 0.0000000000000002
Kappa : 0.761
Mcnemar's Test P-Value : 0.00464
Sensitivity : 0.954
Specificity : 0.784
Pos Pred Value : 0.911
Neg Pred Value : 0.879
Prevalence : 0.700
Detection Rate : 0.667
Detection Prevalence : 0.732
Balanced Accuracy : 0.869
'Positive' Class : Good
Credit_cost and Pricision are in sync.
train results are best for GBM. But its overfitting, i.e. variance is high, so not that great results on test.
test results are best for Random Forest. It has less variance then GBM, but bias is higher.
It may seem like that GBM is a better model, but we still haven’t seen the uncertainity (variance) in the results. Difference between train and test set results give some idea about it, but its better to see it on cross-validated results.
Not much difference here too, DRF seems only slightly better but that may change with fold assignment. For GBM, I did positive class upsample tuning but didn’t tune other hyperparameters. And for DRF I did the exact opposite. So, both the models have a lot of scope of tuning, and I am not at a stage to pick the right model
Important Features
We can see feature importance of either GBM or DRF, but DRF gives a better plot without breaking categorical features into its classes, so we will use DRF.
Topp-3 features are “checking_account_status”, “duration_in_months”, and “credit_amount”
Profiling of best credit-worthy person
To profile a ‘Good’ credit worthy person as per the model, let’s explore the relationship of top predictors with the predicted class for the DRF model.

|
| | 0%
|
|==================================================================================================| 100%
|
| | 0%
|
|==================================================================================================| 100%
`summarise()` ungrouping output (override with `.groups` argument)
The `x` argument of `as_tibble.matrix()` must have unique column names if `.name_repair` is omitted as of tibble 2.0.0.
Using compatibility `.name_repair`.
[90mThis warning is displayed once every 8 hours.[39m
[90mCall `lifecycle::last_warnings()` to see where this warning was generated.[39mIgnoring unknown aesthetics: text
`summarise()` ungrouping output (override with `.groups` argument)
`summarise()` ungrouping output (override with `.groups` argument)
So, the best credit worthy person would have a following profile:
- checking_account_status is “A14” i.e. no checking account
- duration_in_months is less than 12 month i.e. a year
- credit_amount is less than 2k
- credit_history is “A34” i.e. critical account/other existing credits
- Purpose is A43 i.e. radio/television
This seems slightly unintuitive, but I will have to go into model explainibility to get better insights, and currently the time is short for that
Things to do in future
- EDA driven Feature Engineering
- EDA driven Feature Selection
- Better tuning
- with cross validation on credit_cost or Precision
- Bayesian Optimiation
- Model Explainibility
LS0tCnRpdGxlOiAiZXBpRmkgQXNzaWdubWVudCIKYXV0aG9yOiAiU2FoaWwgTWFoZXNod2FyaSIKZGF0ZTogImByIFN5cy5EYXRlKClgIgpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDogCiAgICBkZl9wcmludDogcGFnZWQKICAgIHRvYzogeWVzCiAgICB0b2NfZGVwdGg6IDQKICAgIHRvY19mbG9hdDoKICAgICAgY29sbGFwc2VkOiB5ZXMKICAgICAgc21vb3RoX3Njcm9sbDogeWVzCiAgaHRtbF9ub3RlYm9vazoKICAgIGRmX3ByaW50OiBwYWdlZAogICAgY29kZV9mb2xkaW5nOiAiaGlkZSIKICAgIHRoZW1lOiB1bml0ZWQKICAgIHRvYzogeWVzCiAgICB0b2NfZGVwdGg6IDQKICAgIHRvY19mbG9hdDoKICAgICAgY29sbGFwc2VkOiB5ZXMKICAgICAgc21vb3RoX3Njcm9sbDogeWVzCmVkaXRvcl9vcHRpb25zOgogIGNodW5rX291dHB1dF90eXBlOiBpbmxpbmUKLS0tCgpgYGB7ciwgc2V0dXAsIGluY2x1ZGU9RkFMU0V9Cm9wdGlvbnMoInNjaXBlbiI9MTAwLCAiZGlnaXRzIj00LCAia25pdHIudGFibGUuZm9ybWF0Ij0icGFuZG9jIikKYGBgCgpgYGB7ciwgZ2xvYmFsX29wdGlvbnMsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRSwgZXJyb3IgPSBGQUxTRSwgbWVzc2FnZSA9IFRSVUUsCiAgICAgICAgICAgICAgICAgICAgICBjYWNoZSA9IEZBTFNFLCBjYWNoZS5sYXp5ID0gRkFMU0UsCiAgICAgICAgICAgICAgICAgICAgICB0aWR5ID0gVFJVRSwgaGlnaGxpZ2h0ID0gVFJVRSwgY29sbGFwc2UgPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICAgZmlnLmZ1bGx3aWR0aD1UUlVFLCBmaWcuYWxpZ24gPSAiY2VudGVyIiwgZmlnLndpZHRoID0gMTApCmBgYAoKYGBge3IsIGluY2x1ZGU9RkFMU0V9CnJtKGxpc3QgPSBscygpKQpzb3VyY2UoInV0aWxzLlIiKQpgYGAKCiMgU3RhcnQgIAotIFRoZSBudW1lcmljIGZpbGVzIGhhcyBmZXcgYWRkaXRpb25hbCBmZWF0dXJlcywgYnV0IHRoZXJlIGlzIG5vIGRlc2NyaXB0aW9uIG9mIHRoZSBmZWF0dXJlcyBzbyBJIHdpbGwganVzdCBnbyBhaGVhZCB3aXRoIHRoZSBmaWxlIHdpdGggYm90aCBudW1lcmljIGFuZCBjYXRlZ29yaWNhbCBmZWF0dXJlcwoKCiMgRURBCk5vdGU6IEkgaGF2ZSBvbmx5IGRvbmUgdGhlIEVEQSB0byBhbnN3ZXIgdGhlIGFza2VkIHF1ZXN0aW9ucyBhbmQgdG8gZXhwbG9yZSBzaW1wbGUgYml2YXJpYXRlIHByZWRpY3Rvci10YXJnZXQgcmVsYXRpb25zaGlwcy4gSSBoYXZlIG5vdCBkb25lIGFueSBFREEgZm9yIHRoZSBwdXJwb3NlIG9mIGZlYXR1cmUgZW5naW5lZXJpbmcgb3IgZmVhdHVyZSBzZWxlY3Rpb24uICAKCmBgYHtyLCBpbmNsdWRlPUZBTFNFfQpsb2FkKCJkYXRhX2ludGVybWVkaWF0ZS9kYXRhX2NsZWFuaW5nXzFfb3V0cHV0LlJEYXRhIikKYGBgCgojIyBNaXNzaW5nbmVzcwpgYGB7cn0Kc2FwcGx5KGRhdGFfbWl4LCBmdW5jdGlvbih4KSBzdW0oaXMubmEoeCkpKQpgYGAKClNvLCBubyBtaXNzaW5nIGRhdGEuIFlheXl5IQoKIyMgQ3JlZGl0IFdvcnRoaW5lc3MKQmVmb3JlIGdvaW5nIGludG8gZXhwbG9yaW5nIHJlbGF0aW9uc2hpcCBvZiBwcmVkaWN0b3JzIHdpdGggdGhlIHRhcmdldCwgbGV0J3MgZmlyc3QgY2xlYXJseSBkZWZpbmUgdGhlIHRhcmdldAoKQ3JlZGl0IHdvcnRoaW5lc3MgZm9yIGEgZ3JvdXAgb2Ygb2JzZXJ2YXRpb25zIGNhbiBiZSBtZWFzdXJlZCBieSBHb29kL1RvdGFsIHByb3BvcnRpb24uIEhpZ2hlciB0aGUgcHJvcG9ydGlvbiwgaGlnaGVyIHRoZSBjcmVkaXQgd29ydGhpbmVzcwoKIyMgQ3JlZGl0IEhpc3RvcnkKPiBRdWVzdGlvbjogV291bGQgYSBwZXJzb24gd2l0aCBjcml0aWNhbCBjcmVkaXQgaGlzdG9yeSwgYmUgbW9yZSBjcmVkaXQgd29ydGh5PwoKQWdhaW4sIGxldCdzIGZpcnN0IGRlZmluZSB3aGF0IGNyaXRpY2FsIG1lYW5zLiBJbiB0aGUgYWJzZW5jZSBvZiBhbnkgY29uY3JldGUgZGVmaW5pdGlvbiwgSSB3aWxsIGFzc3VtZSAnY3JpdGljYWwnIHJvdWdobHkgbWVhbnMgbW9yZSBleGlzdGluZyBjcmVkaXRzIGkuZS4gaXQgaW5jcmVhc2UgZnJvbSBBMzAgdG8gQTM1CgpgYGB7ciwgbWVzc2FnZT1GQUxTRX0KcGxvdF9saXN0ID0gcGxvdF90YXJnZXRfY2F0X2ZlYXR1cmVfY2F0KGRhdGFfbWl4LCAiY3JlZGl0X2hpc3RvcnkiLCAiaXNfY3JlZGl0X3dvcnRoeSIsICJHb29kIiwgIkJhZCIpCgpnZ3Bsb3RseSgKICBwbG90X2xpc3RbWzFdXQopCmBgYAoKPiBDcml0aWNhbCBoYXMgcG9zaXRpdmUgYXNzb2NpYXRpb24gd2l0aCBjcmVkaXQgd29ydGhpbmVzcwoKIyMgQWdlCj4gUS4gQXJlIHlvdW5nIHBlb3BsZSBtb3JlIGNyZWRpdHdvcnRoeT8gIAoKYGBge3IsIG1lc3NhZ2U9RkFMU0V9CnBsb3RfbGlzdCA8LSBwbG90X3RhcmdldF9jYXRfZmVhdHVyZV9jb250KGRhdGFfbWl4LCAiYWdlIiwgImlzX2NyZWRpdF93b3J0aHkiKQoKZ2dwbG90bHkoCiAgcGxvdF9saXN0W1sxXV0KKQpgYGAKClRoZSBkaXN0cmlidXRpb25zIGFyZSBxdWl0ZSBvdmVybGFwcGluZy4gQnV0IHRoZXJlIGFyZSBtb3JlIHlvdW5nIGluICJCYWQiIGNvbXBhcmVkIHRvICJHb29kIiwgYW5kIHRoYXQgaXMgYWxzbyB2aXNpYmxlIGluIHRoZSBkaWZmZXJlbmNlIGluIG1lYW5zLiAgCgo+IFNvLCB5b3VuZyBwZW9wbGUgc2VlbSBzbGlnaHRseSBsZXNzIGNyZWRpdCB3b3J0aHkuCgpCdXQgbGV0J3MgYnJlYWsgdGhlIGFnZSBpbnRvIGdyb3VwcyB0byBzZWUgZmluZXIgZGV0YWlscwoKYGBge3IsIGluY2x1ZGU9RkFMU0V9CmRhdGFfbWl4JGFnZV9ncm91cHMgPC0gY3V0KGRhdGFfbWl4JGFnZSwgYnJlYWtzID0gYygxOCwgMjQsIDI5LCAzNCwgMzksIDQ5LCA2NCwgSW5mKSwgb3JkZXJlZF9yZXN1bHQgPSBUKQpgYGAKCmBgYHtyLCBtZXNzYWdlPUZBTFNFfQpwbG90X2xpc3QgPSBwbG90X3RhcmdldF9jYXRfZmVhdHVyZV9jYXQoZGF0YV9taXgsICJhZ2VfZ3JvdXBzIiwgImlzX2NyZWRpdF93b3J0aHkiLCAiR29vZCIsICJCYWQiKQoKZ2dwbG90bHkoCiAgcGxvdF9saXN0W1sxXV0KKQpgYGAKCj4gIkJhZCIgaXMgcXVpdGUgbG93IGZvciB0aGUgKDM0LCAzOV0gYWdlIGdyb3VwCgpgYGB7ciwgaW5jbHVkZT1GQUxTRX0KbG9hZCgiZGF0YV9pbnRlcm1lZGlhdGUvZGF0YV9jbGVhbmluZ18xX291dHB1dC5SRGF0YSIpCmBgYAoKCiMjIENyZWRpdCBBY2NvdW50cwo+IFEuIFdvdWxkIGEgcGVyc29uIHdpdGggbW9yZSBjcmVkaXQgYWNjb3VudHMsIGJlIG1vcmUgY3JlZGl0IHdvcnRoeT8KCkkgYW0gYXNzdW1pbmcgbW9yZSBjcmVkaXQgYWNjb3VudHMgaXMgc2FtZSBhcyAiTnVtYmVyIG9mIGV4aXN0aW5nIGNyZWRpdHMgYXQgdGhpcyBiYW5rIiBpLmUuICdjb3VudF9leGlzdGluZ19jcmVkaXRzJyAKCmBgYHtyLCBpbmNsdWRlPUZBTFNFfQpwbG90X2xpc3QgPC0gcGxvdF90YXJnZXRfY2F0X2ZlYXR1cmVfY2F0KGRhdGFfbWl4LCAiY291bnRfZXhpc3RpbmdfY3JlZGl0cyIsICJpc19jcmVkaXRfd29ydGh5IiwgIkdvb2QiLCAiQmFkIikKYGBgCgpgYGB7ciwgbWVzc2FnZT1GQUxTRX0KZ2dwbG90bHkoCiAgcGxvdF9saXN0W1sxXV0KKQpgYGAKCmBgYHtyfQpnZ3Bsb3RseSgKICBwbG90X2xpc3RbWzJdXQogICwgdG9vbHRpcCA9ICJ0ZXh0IgopCmBgYAoKPiBEYXRhIGlzIHRvbyB1bnJlbGlhYmxlIHRvIHNheSBhbnl0aGluZyBvbiB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gbm8uIG9mIGNyZWRpdCBhY2NvdW50cyBhbmQgY3JlZGl0IHdvcnRoaW5lc3MKCiMjIE90aGVyIGNvbnRpbnVvdXMgZmVhdHVyZXMKSSB3aWxsIGRlZmluZSBjb250aW51b3VzIGFzIGFueSBmYWV0dXJlIGhhdmluZyBtb3JlIHRoYW4gMTAgdW5pcXVlIHZhbHVlcy4gVGhpcyBkZWZpbml0aW9uIHN1aXRzIHRoZSB2aXN1YWxpemF0aW9uIHB1cnBvc2UuCgpgYGB7ciwgaW5jbHVkZT1GQUxTRX0KdGFyZ2V0IDwtICJpc19jcmVkaXRfd29ydGh5IgoKZmVhdHVyZXNfYWxyZWFkeV9jb3ZlcmVkID0gYygiY3JlZGl0X2hpc3RvcnkiLCAiYWdlIiwgImNvdW50X2V4aXN0aW5nX2NyZWRpdHMiKQoKY29udF9jb2x1bW5zIDwtIGNvbG5hbWVzKGRhdGFfbWl4KVtzYXBwbHkoZGF0YV9taXgsIGZ1bmN0aW9uKHgpIG5fZGlzdGluY3QoeCwgbmEucm0gPSBUKSA+IDEwKV0KY29udF9jb2x1bW5zX2xlZnQgPC0gc2V0ZGlmZihjb250X2NvbHVtbnMsIGZlYXR1cmVzX2FscmVhZHlfY292ZXJlZCkKYGBgCgoKYGBge3IsIGluY2x1ZGU9RkFMU0V9CnBsb3RfbGlzdCA9IGxpc3QoKQpmb3IgKGkgaW4gMTpsZW5ndGgoY29udF9jb2x1bW5zX2xlZnQpKSB7CiAgY29udF9jb2x1bW4gPSBjb250X2NvbHVtbnNfbGVmdFtpXQogIAogIHBsb3RfbGlzdF9mb3JDb250VmFyIDwtIHBsb3RfdGFyZ2V0X2NhdF9mZWF0dXJlX2NvbnQoZGF0YV9taXgsIGNvbnRfY29sdW1uLCB0YXJnZXQpCiAgCiAgcGxvdF9saXN0W1tpXV0gPSBwbG90X2xpc3RfZm9yQ29udFZhcltbMV1dCn0KYGBgCgojIyMgVGFyZ2V0IHZzIENvbnRpbnVvdXMgRmVhdHVyZXMgIHsudGFic2V0fQpgYGB7ciwgcmVzdWx0cz0nYXNpcycsIGVjaG8gPSBGQUxTRX0KZm9yIChpIGluIDE6bGVuZ3RoKHBsb3RfbGlzdCkpIHsKICBjYXQoIiMjIyMgIiwgY29udF9jb2x1bW5zX2xlZnRbaV0sICJcbiIpCiAgcHJpbnQocGxvdF9saXN0W1tpXV0pCiAgY2F0KCdcblxuJykKfQpgYGAKCiMjIwpTbywgImR1cmF0aW9uX2luX21vbnRocyIgc2VlbXMgaW1wb3J0YW50LCAiY3JlZGl0X2Ftb3VudCIgbm90CgoKIyMgT3RoZXIgY2F0ZWdvcmljYWwgZmVhdHVyZXMKYGBge3IsIGluY2x1ZGU9RkFMU0V9CmNhdF9jb2x1bW5zIDwtIGNvbG5hbWVzKGRhdGFfbWl4KVtzYXBwbHkoZGF0YV9taXgsIGZ1bmN0aW9uKHgpIG5fZGlzdGluY3QoeCwgbmEucm0gPSBUKSA8PSAxMCldCmNhdF9jb2x1bW5zX2xlZnQgPC0gc2V0ZGlmZihjYXRfY29sdW1ucywgYyhmZWF0dXJlc19hbHJlYWR5X2NvdmVyZWQsIHRhcmdldCkpCmBgYAoKCmBgYHtyLCBpbmNsdWRlPUZBTFNFfQpwbG90X2xpc3QgPSBsaXN0KCkKZm9yIChpIGluIDE6bGVuZ3RoKGNhdF9jb2x1bW5zX2xlZnQpKSB7CiAgY2F0X2NvbHVtbiA9IGNhdF9jb2x1bW5zX2xlZnRbaV0KICAKICBwbG90X2xpc3RfZm9yQ2F0VmFyIDwtIHBsb3RfdGFyZ2V0X2NhdF9mZWF0dXJlX2NhdChkYXRhX21peCwgY2F0X2NvbHVtbiwgdGFyZ2V0LCAiR29vZCIsICJCYWQiKQogIAogIHBsb3RfbGlzdFtbaV1dID0gcGxvdF9saXN0X2ZvckNhdFZhcltbMV1dCn0KYGBgCgojIyMgVGFyZ2V0IHZzIENhdGVnb3JpY2FsIEZlYXR1cmVzICB7LnRhYnNldH0KYGBge3IsIHJlc3VsdHM9J2FzaXMnLCBlY2hvID0gRkFMU0V9CmZvciAoaSBpbiAxOmxlbmd0aChwbG90X2xpc3QpKSB7CiAgY2F0KCIjIyMjICIsIGNhdF9jb2x1bW5zX2xlZnRbaV0sICJcbiIpCiAgcHJpbnQocGxvdF9saXN0W1tpXV0pCiAgY2F0KCdcblxuJykKfQpgYGAKCiMjIwotIENhdC4gZmVhdHVyZXMgd2hpY2ggc2VlbSBkZWZpbml0ZWx5IGltcG9ydGFudDogImNoZWNraW5nX2FjY291bnRfc3RhdHVzIiwgInB1cnBvc2UiLCAicHJlc2VudF9lbXBsb3ltZW50X3NpbmNlIiwgImFzc2V0X3R5cGUiLCAib3RoZXJfaW5zdGFsbG1lbnRfcGxhbnMiLCAiaG91c2luZ190eXBlIiAgCi0gQ2F0LiBmZWF0dXJlcyB3aGVyZSB0aGVyZSBpcyBoaWdoIHVuY2VydGFpbml0eTogInNhdmluZ3NfYWNjb3VudF9zdGF0dXMiLCAiaW5zdGFsbG1lbnRfYXNfcGVyY2VudF9vZl9pbmNvbWUiLCAiaXNfZm9yZWlnbl93b3JrZXIiICAKLSBDYXQuIGZlYXR1cmVzIHdoaWNoIGRvbid0OiAibWFyaXRhbF9zZXhfdHlwZSIsICJyb2xlX2luX290aGVyX2NyZWRpdHMiLCAicHJlc2VudF9yZXNpZGVudF9zaW5jZSIsICJlbXBsb3ltZW50X3R5cGUiLCAiY291bnRfZGVwZW5kZW50cyIsICJoYXNfdGVsZXBob25lIiAgCgoKIyBGZWF0dXJlIEVuZ2luZWVyaW5nICYgU2VsZWN0aW9uICAKQXMgbWVudGlvbmVkIGVhcmxpZXIgSSBkaWRuJ3QgZG8gYW55IEVEQSBmcm9tIGZlYXRyZSBlbmdpbmVlcmluZyBwZXJzcGVjdGl2ZS4gU28sIHRoZXJlIGlzIG5vIGZlYXR1cmUgZW5naW5lZXJpbmcuICAKCkZvciBmZWF0dXJlIHNlbGVjdGlvbiB0b28gSSBoYXZlIG5vdCByZWxpZWQgb24gbXkgRURBIGJlY2F1c2UgYml2YXJhaXRlIGFuYWx5c2lzIGlzIG5vdCBlbm91Z2ggdG8gZmlndXJlIG91dCBpbXBvcnRhbnQgZmVhdHVyZXMuIFRoZXJlIGFyZSBiZXR0ZXIgYXV0b21hdGVkIHRlY2huaXF1ZXMuIEkgaGF2ZSB1c2VkIEJvcnV0YSwgd2hpY2ggSSBoYXZlIGZvdW5kIHRvIGJlIHRoZSBiZXN0IGZlYXR1cmUgc2VsZWN0aW9uIHRlY2huaXF1ZSBhbG1vc3QgYWx3YXlzLiBCZWxvdyBpcyBob3cgdGhlIEJvcnV0YSBwbG90IGxvb2tzIGxpa2U6ICAKYGBge3J9CmxvYWQoIm1vZGVscy9ib3J1dGFfdHJhaW5fb2JqLlJEYXRhIikKCnBsb3QoYm9ydXRhX3RyYWluX29iaiwgeGxhYiA9ICIiLCB4YXh0ID0gIm4iKQpsejwtbGFwcGx5KDE6bmNvbChib3J1dGFfdHJhaW5fb2JqJEltcEhpc3RvcnkpLGZ1bmN0aW9uKGkpCiAgYm9ydXRhX3RyYWluX29iaiRJbXBIaXN0b3J5W2lzLmZpbml0ZShib3J1dGFfdHJhaW5fb2JqJEltcEhpc3RvcnlbLGldKSxpXSkKbmFtZXMobHopIDwtIGNvbG5hbWVzKGJvcnV0YV90cmFpbl9vYmokSW1wSGlzdG9yeSkKTGFiZWxzIDwtIHNvcnQoc2FwcGx5KGx6LG1lZGlhbikpCmF4aXMoc2lkZSA9IDEsbGFzPTIsbGFiZWxzID0gbmFtZXMoTGFiZWxzKSwKICAgICBhdCA9IDE6bmNvbChib3J1dGFfdHJhaW5fb2JqJEltcEhpc3RvcnkpLCBjZXguYXhpcyA9IDAuNykKYGBgCgpTZWxlY3RlZCBmZWF0dXJlcyBhcmU6ICAKYGBge3J9CmxvYWQoImRhdGFfaW50ZXJtZWRpYXRlL2RhdGFfYWZ0ZXJfZnMuUkRhdGEiKQpzZXRkaWZmKGNvbG5hbWVzKGRhdGFfYWZ0ZXJfZnMpLCB0YXJnZXQpCmBgYAoKRmVhdHVyZXMgcmVtb3ZlZCBhcmU6CmBgYHtyfQpzZXRkaWZmKGNvbG5hbWVzKGRhdGFfbWl4KSwgY29sbmFtZXMoZGF0YV9hZnRlcl9mcykpCmBgYAoKKipUaGlzIHBlcmZlY3RseSBtYXRjaGVzIHdpdGggb3VyIEVEQSB0b28qKiAgCgojIE1vZGVsaW5nCiMjIFN0cmF0ZWd5Cj4gSXQgaXMgd29yc2UgdG8gY2xhc3MgYSBjdXN0b21lciBhcyAnR29vZCcgd2hlbiB0aGV5IGFyZSAnQmFkJywgdGhhbiBpdCBpcyB0byBjbGFzcyBhIGN1c3RvbWVyIGFzIGJhZCB3aGVuIHRoZXkgYXJlIGdvb2QuICAKCkxldCAnR29vZCcgYmUgdGhlIHBvc2l0aXZlIGNsYXNzLCBhbmQgJ0JhZCcgYmUgdGhlIG5lZ2F0aXZlIGNsYXNzLiBTbyB0aGUgYWJvdmUgc3RhdGVtZW50IHdpbGwgdHJhbnNsYXRlIHRvOiAgIAoKPiBGYWxzZSBQb3NpdGl2ZXMgKEZQcykgYXJlIG1vcmUgZXhwZW5zaXZlIHRoYW4gRmFsc2UgTmVnYXRpdmVzIChGTnMpIAoKU3VjaCBjYXNlcyBmYWxsIHVuZGVyICoqQ29zdCBTZW5zaXRpdmUgTGVhcm5pbmciIHN0cmF0ZWd5LCBhbmQgZm9sbG93b25nIHN1Yi1zdHJhdGVnaWVzIGNhbiBiZSBmb2xsb3dlZCBkZWNpZGVkIHVuZGVyIGl0OgoKIyMjIFN0cmF0ZWd5IE9wdGlvbnMKLSBNb2RlbGluZyBTdHJhdGVnaWVzIGZvciBjb3N0IHNlbnNpdGl2ZSBsZWFybmluZwogIC0gQ2hhbmdlIGNvc3QgZnVuY3Rpb24KICAgIC0gQ2hhbmdlIHRoZSBmdW5jdGlvbiBpdHNlbGYKICAgICAgLSB0aGUgbWFpbiBmdW5jdGlvbgogICAgICAtIHBlbmFsdHkgY29tcG9uZW50CiAgICAtIENoYW5nZSBmdW5jdGlvbiBwYXJhbWV0ZXJzCiAgICAgIC0gb3ZlcnNhbXBsZSBwb3NpdGl2ZSBjbGFzcwogICAgICAgIC0gc3ludGhldGljIHNhbXBsZSBnZW5lcmF0aW9uIChsaWtlIFNNT1RFKQogICAgICAgIC0gZ2l2ZSBtb3JlIHdlaWdodAogICAgICAtIHVuZGVyc2FtcGxlIHNhbXBsZSBuZWdhdGl2ZSBjbGFzcwogICAgICAgIC0gZ2l2ZSBsZXNzIHdlaWdodAogIC0gT3B0aW1pemUgdGhyZXNob2xkcyB0aGF0IGFyZSB1c2VkIGZvciBjb252ZXJ0aW5nIG91dHB1dCBwcm9iYWJpbGl0aWVzIGludG8gY2xhc3MgbGFiZWxzIC0gdmFsaWQgb25seSBmb3IgbW9kZWxzIHdoaWNoIG91dHB1dCBwcm9iYWJpbGl0aWVzCiAgLSBFbnNlbWJsaW5nCiAgCi0gRXZhbHVhdGlvbiBTdHJhdGVnaWVzIGZvciBDb3N0IHNlbnNpdGl2ZSBjbGFzc2lmaWNhdGlvbgogIC0gRmF2b3VyIFByZWNpc2lvbiBvdmVyIEFjY3VyYWN5IG9yIFJlY2FsbAogIC0gR2l2ZSB3ZWlnaHRzIHRvIGRpZmZlcmVudCBidWNrZXRzIGluIGNvbmZ1c2lvbiBtYXRyaXgsIGFuZCB1c2UgdGhhdCB0byBjb25zdHJ1Y3QgYSBjdXN0b20gZXZhbHVhdGlvbiBtZXRyaWMKCiMjIyBPcHRpb25zIHRoYXQgSSB3aWxsIGV4cGxvcmUKIyMjIE1vZGVscwpJIHdpbGwgdHJ5IHRoZSBmb2xsb3dpbmcgdGhyZWUgbW9kZWxzOgotIExvZ2lzdGljIFJlZ3Jlc3Npb24KLSBCb29zdGVkIFRyZWVzOiBHQk0KLSBSYW5kb20gRm9yZXN0CgojIyMgTW9kZWxpbmcgU3RyYXRlZ3kKLSBXaWxsIG9wdGltaXplIHRocmVzaG9sZHMgZm9yIGFsbCB0aGUgbW9kZWxzCi0gZ2l2ZSBtb3JlIHdlaWdodCB0byBwb3NpdGl2ZSBjbGFzcywgSSB3aWxsIHR1bmUgdGhlIHdlaWdoaW5nIHBhcmFtZXRlcjogd2lsbCBkbyB0aGlzIG9ubHkgZm9yIEdCTSwganVzdCB0byBzaG93Y2FzZQogIAojIyMgRXZhbHVhdGlvbiBTdHJhdGVneQpJIHdpbGwgZ28gd2l0aCBhIEN1c3RvbSBldmFsdWF0aW9uIG1ldHJpYzoKICAKYGBge3IsIGluY2x1ZGU9RkFMU0V9CmxvYWQoImRhdGFfaW50ZXJtZWRpYXRlL2Nvc3RzLlJEYXRhIikKYGBgCgpJIGhhdmUgYXNzaWduZWQgZm9sbHdpbmcgd2VpZ2h0cyB0byBkaWZmZXJlbnQgYnVja2V0cyBvZiB0aGUgY29uZnVzaW9uIG1hdHJpeCB0byBwZW5hbGl6ZSBlYWNoIGJ1Y2tldCBkaWZmZXJlbnRseQpgYGB7cn0KcHJpbnQoY29zdHMpCmBgYAoKVGhlcmUgaXMgbm8gcGFydGljdWxhciByZWFzb24gZm9yIHRoZXNlIHZhbHVlcywganVzdCB0aGVpciByZWxhdGl2ZSBkaWZmZXJlbmNlcyBhcmUgaW1wb3J0YW50IGJlY2F1c2UgdGhleSBwZW5hbGl6ZSBGUHMgbW9yZSB0aGFuIEZOcy4gUEx1cywgSSBhbSByZXdhcmRpbmcgVFBzIChUcnVlIFBvc2l0aXZlcykKCk5vdywgdGhlIGN1c3RvbSBtZXRyaWMgaXMganVzdCB0aGUgbm9ybWFsaXplZCBzdW0tcHJvZHVjdCBvZiB0aGVzZSB3ZWlnaHRzIGFuZCB0aGUgY29uZnVzaW9uIG1hdHJpeCBvZiB0aGUgbW9kZWwuIExldCdzIGNhbGwgaXQgImNyZWRpdF9jb3N0Ii4KCiMjIFNwbGl0dGluZwpJIGhhdmUgODA6MjAgc3BsaXR0aW5nLiBGb3IgdmFsaWRhdGlvbiwgSSB3aWxsIGJlIHVzaW5nIGNyb3NzLXZhbGlkYXRpb24gd2hlcmV2ZXIgcmVxdWlyZWQuCgojIyBCYXNlbGluZQpgYGB7cn0KbG9hZCgiZGF0YV9pbnRlcm1lZGlhdGUvc3BsaXR0ZWRfZGF0YS5SRGF0YSIpCmBgYAoKSSBhbSB0YWtpbmcgYmFzZWxpbmUgYXMgcHJlZGljdGluZyBldmVyeWJvZHkgYXMgIkdvb2QnCgpUcmFpbiBjcmVkaXRfY29zdApgYGB7cn0KdHJhaW5fdHJ1dGhfdGFibGUgPSB0YWJsZShkYXRhX3RyYWluJGlzX2NyZWRpdF93b3J0aHkpCgojIGdpdmUgbG9hbiB0byBldmVyeWJvZHkKYmFzZWxpbmVfdHJhaW5fY29zdCA9IGFzLm51bWVyaWMoKHRyYWluX3RydXRoX3RhYmxlWydHb29kJ10gKiBjb3N0c1sxLCAxXSArIHRyYWluX3RydXRoX3RhYmxlWydCYWQnXSAqIGNvc3RzWzEsIDJdKSAvIG5yb3coZGF0YV90cmFpbikpCmJhc2VsaW5lX3RyYWluX3ByZWNpc2lvbiA9IHRyYWluX3RydXRoX3RhYmxlWydHb29kJ10vbnJvdyhkYXRhX3RyYWluKQptZXNzYWdlKCJCYXNlbGluZSBUcmFpbiBDb3N0OiAiLCBiYXNlbGluZV90cmFpbl9jb3N0LCAiXG5CYXNlbGluZSBUcmFpbiBQcmVjaXNpb246ICIsIGJhc2VsaW5lX3RyYWluX3ByZWNpc2lvbikKYGBgCgpUZXN0IGNyZWRpdF9jb3N0CmBgYHtyfQp0ZXN0X3RydXRoX3RhYmxlID0gdGFibGUoZGF0YV90ZXN0JGlzX2NyZWRpdF93b3J0aHkpCgojIGdpdmUgbG9hbiB0byBldmVyeWJvZHkKYmFzZWxpbmVfdGVzdF9jb3N0ID0gYXMubnVtZXJpYygodGVzdF90cnV0aF90YWJsZVsnR29vZCddICogY29zdHNbMSwgMV0gKyB0ZXN0X3RydXRoX3RhYmxlWydCYWQnXSAqIGNvc3RzWzEsIDJdKSAvIG5yb3coZGF0YV90ZXN0KSkKYmFzZWxpbmVfdGVzdF9wcmVjaXNpb24gPSB0ZXN0X3RydXRoX3RhYmxlWydHb29kJ10vbnJvdyhkYXRhX3Rlc3QpCm1lc3NhZ2UoIkJhc2VsaW5lIFRlc3QgQ29zdDogIiwgYmFzZWxpbmVfdGVzdF9jb3N0LCAiXG5CYXNlbGluZSBUZXN0IFByZWNpc2lvbjogIiwgYmFzZWxpbmVfdGVzdF9wcmVjaXNpb24pCmBgYAoKCiMjIExvZ2lzdGljIFJlZ3Jlc3Npb24KSSBoYXZlIGp1c3QgdHJhaW5lZCBhIHNpbXBsZSBMb2dpc3RpYyBSZWdyZXNzaXNvbiB3aXRob3V0IGFueSByZWd1bGFyaXphdGlvbiwgY2xhc3Mgd2VpZ2hpbmcsIG9yIGh5cGVycGFyYW1ldGVyIHR1bmluZy4gVGhpcyBpcyBzdXBwb3NlZCB0byBiZSBhIE1MIG1vZGVsIGJhc2VsaW5lLgoKYGBge3IsIGluY2x1ZGU9RkFMU0V9Cmgyby5pbml0KCkKCnRyYWluX2gybyA8LSBhcy5oMm8oZGF0YV90cmFpbikKdGVzdF9oMm8gIDwtIGFzLmgybyhkYXRhX3Rlc3QpCgp5IDwtICJpc19jcmVkaXRfd29ydGh5Igp4IDwtIHNldGRpZmYobmFtZXModHJhaW5faDJvKSwgeSkKYGBgCgpgYGB7ciwgaW5jbHVkZT1GQUxTRX0KbW9kZWxfZ2xtX2Jhc2UgPC0gaDJvOjpoMm8ubG9hZE1vZGVsKCJtb2RlbHMvR0xNX21vZGVsX1JfMTU5NTMwNDQwNTEwNl8xIikKYGBgCgpgYGB7ciwgaW5jbHVkZT1GQUxTRX0KdHJhaW5fcHJlZF9kZl9nbG1fYmFzZSA8LSBnZXRfcHJlZGljdGlvbihtb2RlbF9nbG1fYmFzZSwgdHJhaW5faDJvKQp0cmFpbl9yZXN1bHRzX2dsbSA8LSBnZXRfcmVzdWx0cyh0cmFpbl9wcmVkX2RmX2dsbV9iYXNlLCBjb3N0cykKb3B0X3RocmVzaG9sZF9nbG0gPSBvcHRpbWl6ZShnZXRfY29zdF9naXZlbl90aHJlc2hvbGQsIGMoMC4xLCAwLjkpLCB0cnV0aF9wcmVkX2RmPXRyYWluX3ByZWRfZGZfZ2xtX2Jhc2UsIGNvc3RzPWNvc3RzKQoKdGVzdF9wcmVkX2RmX2dsbV9iYXNlX2RlZmF1bHQgPC0gZ2V0X3ByZWRpY3Rpb24obW9kZWxfZ2xtX2Jhc2UsIHRlc3RfaDJvKQp0ZXN0X3ByZWRfZGZfZ2xtX2Jhc2UgPC0gZ2V0X3ByZWRpY3Rpb25fY2xhc3NfZ2l2ZW5fdGhyZXNob2xkKHRlc3RfcHJlZF9kZl9nbG1fYmFzZV9kZWZhdWx0LCBvcHRfdGhyZXNob2xkX2dsbSRtaW5pbXVtKQp0ZXN0X3Jlc3VsdHNfZ2xtIDwtIGdldF9yZXN1bHRzKHRlc3RfcHJlZF9kZl9nbG1fYmFzZSwgY29zdHMpCmBgYAoKVHJhaW4gUmVzdWx0czogIApgYGB7cn0KdHJhaW5fcmVzdWx0c19nbG0kcmVzdWx0cwpgYGAKClRlc3QgUmVzdWx0czogIApgYGB7cn0KdGVzdF9yZXN1bHRzX2dsbSRyZXN1bHRzCmBgYAoKIyMgQm9vc3RlZCBUcmVlcyAtIEdCTQpIZXJlLCBJIGhhdmUgd2VpZ2hlZCB0aGUgY2xhc3NlcyB0byBmYXZvdXIgIkdvb2QiLiBXZWlnaHQgaGFzIGJlZW4gZm91bmQgYnkgdHVuaW5nIHZpYSBncmlkIHNlYXJjaCB1c2luZyA1LWZvbGQgY3Jvc3MgdmFsaWRhdGlvbi4gSSBoYXZlIG5vdCB0dW5lZCBvdGhlciBoeXBlcnBhcmFtZXRlcnMuICAKCmBgYHtyLCBpbmNsdWRlPUZBTFNFfQptb2RlbF9nYm1fYmVzdCA8LSBoMm8ubG9hZE1vZGVsKCJtb2RlbHMvZ2JtX2dyaWRfMTFfbW9kZWxfMyIpCmBgYAoKYGBge3IsIGluY2x1ZGU9RkFMU0V9CnRyYWluX3ByZWRfZGZfZ2JtX2Jhc2UgPC0gZ2V0X3ByZWRpY3Rpb24obW9kZWxfZ2JtX2Jlc3QsIHRyYWluX2gybykKdHJhaW5fcmVzdWx0c19nYm0gPC0gZ2V0X3Jlc3VsdHModHJhaW5fcHJlZF9kZl9nYm1fYmFzZSwgY29zdHMpCm9wdF90aHJlc2hvbGRfZ2JtID0gb3B0aW1pemUoZ2V0X2Nvc3RfZ2l2ZW5fdGhyZXNob2xkLCBjKDAuMSwgMC45KSwgdHJ1dGhfcHJlZF9kZj10cmFpbl9wcmVkX2RmX2dibV9iYXNlLCBjb3N0cz1jb3N0cykKCnRlc3RfcHJlZF9kZl9nYm1fYmFzZV9kZWZhdWx0IDwtIGdldF9wcmVkaWN0aW9uKG1vZGVsX2dibV9iZXN0LCB0ZXN0X2gybykKdGVzdF9wcmVkX2RmX2dibV9iYXNlIDwtIGdldF9wcmVkaWN0aW9uX2NsYXNzX2dpdmVuX3RocmVzaG9sZCh0ZXN0X3ByZWRfZGZfZ2JtX2Jhc2VfZGVmYXVsdCwgb3B0X3RocmVzaG9sZF9nYm0kbWluaW11bSkKdGVzdF9yZXN1bHRzX2dibSA8LSBnZXRfcmVzdWx0cyh0ZXN0X3ByZWRfZGZfZ2JtX2Jhc2UsIGNvc3RzKQpgYGAKClRyYWluIFJlc3VsdHM6ICAKYGBge3J9CnRyYWluX3Jlc3VsdHNfZ2JtJHJlc3VsdHMKYGBgCgpUZXN0IFJlc3VsdHM6ICAKYGBge3J9CnRlc3RfcmVzdWx0c19nYm0kcmVzdWx0cwpgYGAKCgojIyBSYW5kb20gRm9yZXN0CkhlcmUsIEkgaGF2ZSB0dW5lZCBvdGhlciBoeXBlcnBhcmFtZXRlcnMgdmlhIHJhbmRvbSBzZWFyY2ggdXNpbmcgNS1mb2xkIGNyb3NzIHZhbGlkYXRpb24uIEkgaGF2ZSBub3QgZG9uZSBjbGFzcyB3ZWlnaGluZyBoZXJlLiAgCgpgYGB7ciwgaW5jbHVkZT1GQUxTRX0KaDJvLmxvYWRHcmlkKAogICJtb2RlbHMvZHJmX2dyaWRfMTEiCikKCmRyZl9ncmlkcGVyZl8yX2J5UHJlY2lzaW9uIDwtIGgyby5nZXRHcmlkKGdyaWRfaWQgPSAiZHJmX2dyaWRfMTEiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzb3J0X2J5ID0gInByZWNpc2lvbiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRlY3JlYXNpbmcgPSBUUlVFKQoKZHJmX2dyaWRwZXJmXzJfYnlBVUMgPC0gaDJvLmdldEdyaWQoZ3JpZF9pZCA9ICJkcmZfZ3JpZF8xMSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNvcnRfYnkgPSAiYXVjIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGVjcmVhc2luZyA9IFRSVUUpCgojIEdyYWIgdGhlIHRvcCBHQk0gbW9kZWwsIGNob3NlbiBieSB2YWxpZGF0aW9uIEFVQwptb2RlbF9kcmZfYmVzdCA8LSBoMm8uZ2V0TW9kZWwoZHJmX2dyaWRwZXJmXzJfYnlBVUNAbW9kZWxfaWRzW1sxXV0pCmBgYAoKYGBge3IsIGluY2x1ZGU9RkFMU0V9CnRyYWluX3ByZWRfZGZfZHJmX2Jhc2UgPC0gZ2V0X3ByZWRpY3Rpb24obW9kZWxfZHJmX2Jlc3QsIHRyYWluX2gybykKdHJhaW5fcmVzdWx0c19kcmYgPC0gZ2V0X3Jlc3VsdHModHJhaW5fcHJlZF9kZl9kcmZfYmFzZSwgY29zdHMpCm9wdF90aHJlc2hvbGRfZHJmID0gb3B0aW1pemUoZ2V0X2Nvc3RfZ2l2ZW5fdGhyZXNob2xkLCBjKDAuMSwgMC45KSwgdHJ1dGhfcHJlZF9kZj10cmFpbl9wcmVkX2RmX2RyZl9iYXNlLCBjb3N0cz1jb3N0cykKCnRlc3RfcHJlZF9kZl9kcmZfYmFzZV9kZWZhdWx0IDwtIGdldF9wcmVkaWN0aW9uKG1vZGVsX2RyZl9iZXN0LCB0ZXN0X2gybykKdGVzdF9wcmVkX2RmX2RyZl9iYXNlIDwtIGdldF9wcmVkaWN0aW9uX2NsYXNzX2dpdmVuX3RocmVzaG9sZCh0ZXN0X3ByZWRfZGZfZHJmX2Jhc2VfZGVmYXVsdCwgb3B0X3RocmVzaG9sZF9kcmYkbWluaW11bSkKdGVzdF9yZXN1bHRzX2RyZiA8LSBnZXRfcmVzdWx0cyh0ZXN0X3ByZWRfZGZfZHJmX2Jhc2UsIGNvc3RzKQpgYGAKClRyYWluIFJlc3VsdHM6ICAKYGBge3J9CnRyYWluX3Jlc3VsdHNfZHJmJHJlc3VsdHMKYGBgCgpUZXN0IFJlc3VsdHM6ICAKYGBge3J9CnRlc3RfcmVzdWx0c19kcmYkcmVzdWx0cwpgYGAKCiMjIENvbXBhcmlzb24KYGBge3J9Cm1vZGVsX2NvbXBhcmlzb25fZGYgPC0gZGF0YS5mcmFtZSgKICBtb2RlbHMgPSBjKCJiYXNlbGluZSIsICJMb2dpc3RpYyBSZWdyZXNzaW9uIiwgIkdCTSIsICJSYW5kb20gRm9yZXN0IiksCiAgdHJhaW5fY3JlZGl0X2Nvc3QgPSBjKGJhc2VsaW5lX3RyYWluX2Nvc3QsIG9wdF90aHJlc2hvbGRfZ2xtJG9iamVjdGl2ZSwgb3B0X3RocmVzaG9sZF9nYm0kb2JqZWN0aXZlLCBvcHRfdGhyZXNob2xkX2RyZiRvYmplY3RpdmUpLAogIHRyYWluX3ByZWNpc2lvbiA9IGMoYmFzZWxpbmVfdHJhaW5fcHJlY2lzaW9uLCB0cmFpbl9yZXN1bHRzX2dsbSRyZXN1bHRzJGJ5Q2xhc3NbIlBvcyBQcmVkIFZhbHVlIl0sIHRyYWluX3Jlc3VsdHNfZ2JtJHJlc3VsdHMkYnlDbGFzc1siUG9zIFByZWQgVmFsdWUiXSwgdHJhaW5fcmVzdWx0c19kcmYkcmVzdWx0cyRieUNsYXNzWyJQb3MgUHJlZCBWYWx1ZSJdKSwKICB0ZXN0X2NyZWRpdF9jb3N0ID0gYyhiYXNlbGluZV90ZXN0X2Nvc3QsIHRlc3RfcmVzdWx0c19nbG0kY29zdCwgdGVzdF9yZXN1bHRzX2dibSRjb3N0LCB0ZXN0X3Jlc3VsdHNfZHJmJGNvc3QpLAogIHRlc3RfcHJlY2lzaW9uID0gYyhiYXNlbGluZV90ZXN0X3ByZWNpc2lvbiwgdGVzdF9yZXN1bHRzX2dsbSRyZXN1bHRzJGJ5Q2xhc3NbIlBvcyBQcmVkIFZhbHVlIl0sIHRlc3RfcmVzdWx0c19nYm0kcmVzdWx0cyRieUNsYXNzWyJQb3MgUHJlZCBWYWx1ZSJdLCB0ZXN0X3Jlc3VsdHNfZHJmJHJlc3VsdHMkYnlDbGFzc1siUG9zIFByZWQgVmFsdWUiXSkKKQoKbW9kZWxfY29tcGFyaXNvbl9kZgpgYGAKCj4gQ3JlZGl0X2Nvc3QgYW5kIFByaWNpc2lvbiBhcmUgaW4gc3luYy4KCj4gdHJhaW4gcmVzdWx0cyBhcmUgYmVzdCBmb3IgR0JNLiBCdXQgaXRzIG92ZXJmaXR0aW5nLCBpLmUuIHZhcmlhbmNlIGlzIGhpZ2gsIHNvIG5vdCB0aGF0IGdyZWF0IHJlc3VsdHMgb24gdGVzdC4KCj4gdGVzdCByZXN1bHRzIGFyZSBiZXN0IGZvciBSYW5kb20gRm9yZXN0LiBJdCBoYXMgbGVzcyB2YXJpYW5jZSB0aGVuIEdCTSwgYnV0IGJpYXMgaXMgaGlnaGVyLgoKCkl0IG1heSBzZWVtIGxpa2UgdGhhdCBHQk0gaXMgYSBiZXR0ZXIgbW9kZWwsIGJ1dCB3ZSBzdGlsbCBoYXZlbid0IHNlZW4gdGhlIHVuY2VydGFpbml0eSAodmFyaWFuY2UpIGluIHRoZSByZXN1bHRzLiBEaWZmZXJlbmNlIGJldHdlZW4gdHJhaW4gYW5kIHRlc3Qgc2V0IHJlc3VsdHMgZ2l2ZSBzb21lIGlkZWEgYWJvdXQgaXQsIGJ1dCBpdHMgYmV0dGVyIHRvIHNlZSBpdCBvbiBjcm9zcy12YWxpZGF0ZWQgcmVzdWx0cy4KCmBgYHtyfQp0ZW1wIDwtIG1vZGVsX2dibV9iZXN0QG1vZGVsJGNyb3NzX3ZhbGlkYXRpb25fbWV0cmljc19zdW1tYXJ5CnRlbXBfMSA8LSB0aWJibGUobWV0cmljcyA9IHJvdy5uYW1lcyh0ZW1wKSkgJT4lIAogIGJpbmRfY29scyh0ZW1wICU+JSBhc190aWJibGUoKSkKZGF0YXRhYmxlKHRlbXBfMVssMTozXSwgcm93bmFtZXMgPSBGQUxTRSwgZmlsdGVyPSJub25lIiwgb3B0aW9ucyA9IGxpc3Qoc2Nyb2xsWD1ULCBzY3JvbGxZPSIyMDBweCIsIHNjcm9sbENvbGxhcHNlPUZBTFNFLCBwYWdpbmc9RkFMU0UpKQpgYGAKCmBgYHtyfQp0ZW1wIDwtIG1vZGVsX2RyZl9iZXN0QG1vZGVsJGNyb3NzX3ZhbGlkYXRpb25fbWV0cmljc19zdW1tYXJ5CnRlbXBfMSA8LSBkYXRhLmZyYW1lKG1ldHJpY3MgPSByb3cubmFtZXModGVtcCkpICU+JSAKICBiaW5kX2NvbHModGVtcCAlPiUgYXNfdGliYmxlKCkpCmRhdGF0YWJsZSh0ZW1wXzFbLDE6M10sIHJvd25hbWVzID0gRkFMU0UsIGZpbHRlcj0ibm9uZSIsIG9wdGlvbnMgPSBsaXN0KHNjcm9sbFg9VCwgc2Nyb2xsWT0iMjAwcHgiLCBzY3JvbGxDb2xsYXBzZT1GQUxTRSwgcGFnaW5nPUZBTFNFKSkKYGBgCgpOb3QgbXVjaCBkaWZmZXJlbmNlIGhlcmUgdG9vLCBEUkYgc2VlbXMgb25seSBzbGlnaHRseSBiZXR0ZXIgYnV0IHRoYXQgbWF5IGNoYW5nZSB3aXRoIGZvbGQgYXNzaWdubWVudC4gRm9yIEdCTSwgSSBkaWQgcG9zaXRpdmUgY2xhc3MgdXBzYW1wbGUgdHVuaW5nIGJ1dCBkaWRuJ3QgdHVuZSBvdGhlciBoeXBlcnBhcmFtZXRlcnMuIEFuZCBmb3IgRFJGIEkgZGlkIHRoZSBleGFjdCBvcHBvc2l0ZS4gU28sIGJvdGggdGhlIG1vZGVscyBoYXZlIGEgbG90IG9mIHNjb3BlIG9mIHR1bmluZywgYW5kIEkgYW0gbm90IGF0IGEgc3RhZ2UgdG8gcGljayB0aGUgcmlnaHQgbW9kZWwKCiMgSW1wb3J0YW50IEZlYXR1cmVzCldlIGNhbiBzZWUgZmVhdHVyZSBpbXBvcnRhbmNlIG9mIGVpdGhlciBHQk0gb3IgRFJGLCBidXQgRFJGIGdpdmVzIGEgYmV0dGVyIHBsb3Qgd2l0aG91dCBicmVha2luZyBjYXRlZ29yaWNhbCBmZWF0dXJlcyBpbnRvIGl0cyBjbGFzc2VzLCBzbyB3ZSB3aWxsIHVzZSBEUkYuCgpgYGB7cn0KaDJvLnZhcmltcF9wbG90KG1vZGVsX2RyZl9iZXN0KQpgYGAKCj4gVG9wcC0zIGZlYXR1cmVzIGFyZSAiY2hlY2tpbmdfYWNjb3VudF9zdGF0dXMiLCAiZHVyYXRpb25faW5fbW9udGhzIiwgYW5kICJjcmVkaXRfYW1vdW50IgoKIyBQcm9maWxpbmcgb2YgYmVzdCBjcmVkaXQtd29ydGh5IHBlcnNvbgpUbyBwcm9maWxlIGEgJ0dvb2QnIGNyZWRpdCB3b3J0aHkgcGVyc29uIGFzIHBlciB0aGUgbW9kZWwsIGxldCdzIGV4cGxvcmUgdGhlIHJlbGF0aW9uc2hpcCBvZiB0b3AgcHJlZGljdG9ycyB3aXRoIHRoZSBwcmVkaWN0ZWQgY2xhc3MgZm9yIHRoZSBEUkYgbW9kZWwuCgoKYGBge3IsIGluY2x1ZGU9RkFMU0V9CnRyYWluX3ByZWRfZGZfZHJmX2RlZmF1bHQgPC0gZ2V0X3ByZWRpY3Rpb24obW9kZWxfZHJmX2Jlc3QsIHRyYWluX2gybykKdHJhaW5fcHJlZF9kZl9kcmZfb3B0IDwtIGdldF9wcmVkaWN0aW9uX2NsYXNzX2dpdmVuX3RocmVzaG9sZCh0cmFpbl9wcmVkX2RmX2RyZl9kZWZhdWx0LCBvcHRfdGhyZXNob2xkX2RyZiRtaW5pbXVtKQoKdGVzdF9wcmVkX2RmX2RyZl9kZWZhdWx0IDwtIGdldF9wcmVkaWN0aW9uKG1vZGVsX2RyZl9iZXN0LCB0ZXN0X2gybykKdGVzdF9wcmVkX2RmX2RyZl9vcHQgPC0gZ2V0X3ByZWRpY3Rpb25fY2xhc3NfZ2l2ZW5fdGhyZXNob2xkKHRlc3RfcHJlZF9kZl9kcmZfZGVmYXVsdCwgb3B0X3RocmVzaG9sZF9kcmYkbWluaW11bSkKCmFsbF9wcmVkX2RmX2RyZl9vcHQgPC0gYmluZF9yb3dzKHRyYWluX3ByZWRfZGZfZHJmX29wdCwgdGVzdF9wcmVkX2RmX2RyZl9vcHQpCmBgYAoKYGBge3IsIG1lc3NhZ2U9RkFMU0V9CnBsb3RfbGlzdCA9IHBsb3RfdGFyZ2V0X2NhdF9mZWF0dXJlX2NhdChhbGxfcHJlZF9kZl9kcmZfb3B0LCAiY2hlY2tpbmdfYWNjb3VudF9zdGF0dXMiLCAiaXNfY3JlZGl0X3dvcnRoeV9wcmVkX2NsYXNzIiwgIkdvb2QiLCAiQmFkIikKCmdncGxvdGx5KAogIHBsb3RfbGlzdFtbMV1dCikKYGBgCgpgYGB7ciwgbWVzc2FnZT1GQUxTRX0KcGxvdF9saXN0ID0gcGxvdF90YXJnZXRfY2F0X2ZlYXR1cmVfY29udChhbGxfcHJlZF9kZl9kcmZfb3B0LCAiZHVyYXRpb25faW5fbW9udGhzIiwgImlzX2NyZWRpdF93b3J0aHlfcHJlZF9jbGFzcyIpCgpnZ3Bsb3RseSgKICBwbG90X2xpc3RbWzFdXQopCmBgYAoKYGBge3IsIG1lc3NhZ2U9RkFMU0V9CnBsb3RfbGlzdCA9IHBsb3RfdGFyZ2V0X2NhdF9mZWF0dXJlX2NvbnQoYWxsX3ByZWRfZGZfZHJmX29wdCwgImNyZWRpdF9hbW91bnQiLCAiaXNfY3JlZGl0X3dvcnRoeV9wcmVkX2NsYXNzIikKCmdncGxvdGx5KAogIHBsb3RfbGlzdFtbMV1dCikKYGBgCgpgYGB7ciwgbWVzc2FnZT1GQUxTRX0KcGxvdF9saXN0ID0gcGxvdF90YXJnZXRfY2F0X2ZlYXR1cmVfY2F0KGFsbF9wcmVkX2RmX2RyZl9vcHQsICJjcmVkaXRfaGlzdG9yeSIsICJpc19jcmVkaXRfd29ydGh5X3ByZWRfY2xhc3MiLCAiR29vZCIsICJCYWQiKQoKZ2dwbG90bHkoCiAgcGxvdF9saXN0W1sxXV0KKQpgYGAKCmBgYHtyLCBtZXNzYWdlPUZBTFNFfQpwbG90X2xpc3QgPSBwbG90X3RhcmdldF9jYXRfZmVhdHVyZV9jYXQoYWxsX3ByZWRfZGZfZHJmX29wdCwgInB1cnBvc2UiLCAiaXNfY3JlZGl0X3dvcnRoeV9wcmVkX2NsYXNzIiwgIkdvb2QiLCAiQmFkIikKCmdncGxvdGx5KAogIHBsb3RfbGlzdFtbMV1dCikKYGBgCgpTbywgdGhlIGJlc3QgY3JlZGl0IHdvcnRoeSBwZXJzb24gd291bGQgaGF2ZSBhIGZvbGxvd2luZyBwcm9maWxlOiAgCi0gY2hlY2tpbmdfYWNjb3VudF9zdGF0dXMgaXMgIkExNCIgaS5lLiBubyBjaGVja2luZyBhY2NvdW50ICAKLSBkdXJhdGlvbl9pbl9tb250aHMgaXMgbGVzcyB0aGFuIDEyIG1vbnRoIGkuZS4gYSB5ZWFyICAKLSBjcmVkaXRfYW1vdW50IGlzIGxlc3MgdGhhbiAyayAgCi0gY3JlZGl0X2hpc3RvcnkgaXMgIkEzNCIgaS5lLiBjcml0aWNhbCBhY2NvdW50L290aGVyIGV4aXN0aW5nIGNyZWRpdHMgIAotIFB1cnBvc2UgaXMgQTQzIGkuZS4gcmFkaW8vdGVsZXZpc2lvbiAgCgpUaGlzIHNlZW1zIHNsaWdodGx5IHVuaW50dWl0aXZlLCBidXQgSSB3aWxsIGhhdmUgdG8gZ28gaW50byBtb2RlbCBleHBsYWluaWJpbGl0eSB0byBnZXQgYmV0dGVyIGluc2lnaHRzLCBhbmQgY3VycmVudGx5IHRoZSB0aW1lIGlzIHNob3J0IGZvciB0aGF0CgojIFRoaW5ncyB0byBkbyBpbiBmdXR1cmUgIAotIEVEQSBkcml2ZW4gRmVhdHVyZSBFbmdpbmVlcmluZyAgCi0gRURBIGRyaXZlbiBGZWF0dXJlIFNlbGVjdGlvbiAgCi0gQmV0dGVyIHR1bmluZyAgCiAgLSB3aXRoIGNyb3NzIHZhbGlkYXRpb24gb24gY3JlZGl0X2Nvc3Qgb3IgUHJlY2lzaW9uICAKICAtIEJheWVzaWFuIE9wdGltaWF0aW9uICAKLSBNb2RlbCBFeHBsYWluaWJpbGl0eSAg